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Tho flv« protocol m«t«rl«l« units rvaluAtcd In thl« report ver« 
developed by the Protocol Materlrls Development Project, University of 
ColomdOi under a grant fron the Drpartnent of Healtht Education and 
Welfare, United States Office of Education, 0EC-O-70-404S (725). The 
period of fundlne for the research* devr.loptnent , testing, and evaluation 
of these products vas from July 1, 1970 through December 31, 1972 « The 
total amount of the grants for that period vas $105,604* A rou^h, total 
cost per product from beginning of development to an evaluated, finished 
product, with multiple reprint capacity, averages about $21,000. The 
average actual production cost, not Including salaries and Indirect coats, 
VAS about $6,000 for each product. 

The five products Illustrate concepts Important In the preparation 
of teachers. The concepts were selected and developed vlth reference to 
the basic Pedagogical Plan outlined In the USOE Requcrit for Proposal In 
April 1970. The five products In order of their development are: 

1. ( ;9ncept ua llislng the Process Qf I nstyuctloa (One 10«-inlnute 

16 mm. black and white sound film; teacher *a guide; student 
handout,) 

2. learnqr** .f»Dd Thctr jCI)jirnrtexi«f ImnlixaLlon^. for Tnatjruc- 

tlona^ pe clslon -Malt jnfc (Two sets of color slides, "Attitudes 
Toward School" and ''Instructional Alt rnntivcs"; casfinttr tape 
to accompany "Instructional Alr.eniativeft" ; teacher *a culde; 
student profice booklets.) 

3 • V fy^al Intera c tion _Ln_J:hc_£a£n itj ve Dlmcnj^lon;. JTho Relation- 
f h^PB B et Wf* n_ T e a c h e . V e r ^> a j ^^ehnvior^nc* Student Hcgijonso 
(One IS^mlnute 16nm. black and white sound film; teacher's 
guide; student handout.) 

4. Qrganlgl nA Facts to T ea ch Mya n lnRful Relationships (One 
14-mlnute 16nn* color sound film; teacher *8 guide; student 
handout •) 

5. Fyjr Verbal Behavi or (One 14*-mlnute 16 nn. color sound film; 
teacher^s guide; student handout.) 
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EVALUATION REPORT 
OP THE 1970-72 
PROTOCOL M/\TERIALS UNITS 
DEVELOPED BY THE 
PROTOCOL MATERIALS DEVELOPMLNT PROJECT 
UNIVERSITY OF COLORADO 



RATIONALE AND OBJECTIVES FOR EVALUATION 

Evciluatlon Is generally defined as the aystematlc collection, 
analysis and interpretation of data for the purposes of detenalning 
the effectiveness and worth of an educational progran or product, and 
aiding the developers of program or product to make more intelligent 
decisions concerning its revisions or implementations, :valuation 
should consist of descriptions of the inputs, processes and outcomes, 
as well as of judgements of various compononts of the program or 
product. 

The developers of the University of Colorado Protocol Materials 
Project asked two questions which served to direct the evaluation study. 
1. How effective is each of the five products?"* 

Five criteria were used for judging the effectiveness: 
interest, significance, clarity, sufficiency, and perceived 
effectiveness. The criteria were assessed by the evaluation 
questionnaires, the criterion tests, and the post-tests. 



* Conceptualizing the Process o f Instruction . lear ners and Their 
Chnract eristics; Implications for Ins tructiona l Deci . ion-Mak ing , 
Verbal In t l, taction in the Cor.nltiv e nlmcn alonr The Rcl ationship^ I^ctvee n 
% Teacher >er ba l Behavior and Stu dent jUjf^j>om^c« O rpanizinr, Facts to Tc?ach 

* Meaningful Relat ionships, and Fa ir Verbal behavior . 
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2* Is the University of Colorado Protocol Materials approach 
different from conventional approaches In teacher education courses? 

Two criteria were used for judging the difference: interest 
and worthwhlleness, as Indicated by students and Instructors on 
the Evaluation Questionnaires. 



PROCEDURES FOR CONDUCTING FirXD TEST 

The evaluation study was conducted as part of an extensive field 
test of the materials In various teacher education institutions around 
the country. 

Selection of Field Test Instructors and Students 

It was determined by the project staff that the materials should he 
tested in both social science methods classes and in general methods clnnses 
in undergraduate teacher preparation programs in colleges and univeraltles 
in different geographical regions of the United Sta»:es and in both urban 
and rural areas. Social science methods classes were selected because the 
classroom episodes used in the films and slide-tapes aro all set in social 
studies classrooms. Further, the academic back^^ound of the projdect staff 
is largely in the social sciences, and the concepts included in the five 
products were selected because of their presumed relevance to successful 
teaching and learning in secondary social studies classes. The staff was 
Interested in finding out differences in effects of the protocol materl/^ls 
on students in general methods classes and in social science methods classes. 

The director of the project contacted key educators in the various 
geographic rej»lons and asked for names of Instructors in colleges and 
universities in that general area who might be Interested in participating 
in the field test. Telephone calls were made to about twenty persons and 
from those five social science metl ds Instructors and five general methods 
instructors were selected for the experimental grovps. Of those ten the 



-4- 



director was personally acqaalnted with three t The names and Institutional 
affiliation of the ten instructors appear In Appendix a. 

All instructs ns and materials were sent by Air Express to the in"- 
structors during the last week In March » 1972. For every Instrument admin- 
istered to the students 9 a large, stamped return envelope was provided. 
The instruments or questiornaires were to be returned to the project staff 
as soon as they had been administered. Instructors in the experimental 
classes had the prerogative to decide when and In what order to teach the 
protocol units. Some cla$;sc8 were on the semester system which brought 
the materials into the middle of the semester jsnd some classes were just 
beginning the spring quarter. 

Sele ct ion of the Comparison Group 

The comparison or control group consisted of five social science 
methods classes. The project staff assumed that students in social science 
methods classes would already have had general methods classes and v/ould 
probably have a more sophisticated under standing of the concepts employed 
In the protocol units than the general methods students. The selection of 
such classes for control would give a fairer test of the five products. The 
Instructors were selected by the director from among persons suggested by 
educators at universities in different parts of the country. Three of the 
five control Instructors were acquainted with the project director. A list 
of control instructors and their Institutions appears in Appendix A. 

Mailing of Instructions, background questionnaires » and pre-tests took 
place in late March. Stamped return envelopes were provided for mailing 
materials back to the project stalf . Post-tests were mailed in early May. 

EKLC 
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The jjistrmnents , Collection , and Treatment of the Data 

The instrtmenis used to collect the data were constructed by the 
University of Colorado Protocol Materials Development staff according to 
the objectives of tho. project and In order to answer the two evaluative 
questions I "How eff<?ctive is each product?" and "is the Protocol approach 
different from conventional approaches In teacher education courses?" 

A* "How effective Is each product?" 

Five criteria were mentioned in the development proposal of the 
University of Colorado Protocol Materials Development Project. These are: 
significance, interest, sufficiency, clarity, and effectiveness. These 
criteria were assessed by Student and Instructor Evaluation Questionnaires 
administered after use of each protocol unit. The criteria were defined 
and assessed in the following ways: 

Significance has two aspects: 1.) the importance of the ideas tauglit 
in the protocol units; and 2.) the Importance and worth of the activities 
Irtroduced to teach these ideas. The Student and Instructor Evaluation 
Questionnaires, administered at the end of each unit, contained these 
questions which measured significance on a Llkert scale. 

The concepts dealt with in this protocol are worthwhile 
for teacher preparation. 

SA A N D SD 
The discussion following the film (or slides) was not very 
significant* 

SA A N D SD 

Interest was usually expressed in comparative terms on the Student 
and Instructor Evaluation Questionnaires, The following are examples 




* See Appendix B 
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of items used to assess the degree of Interest In the materials. 

Compared to what normally takes place in my education courses » 
this protocol unit was more interesting. 

SA A N D SD 
This color film was better and more interesting than the 
same film would have been in black and white. 

SA A N D SD 

Sufficiency refers to the adequacy of the protocol in general, 
and more specifically to the adequacy of the separate component parts 
such as the quality of the films, the acting in the sequences, or the 
organization of activities. Examples of items which assessed this 
criterion are the following; 

This protocol unit meeds major revision before further use. 

SA A N D SD 
(Instructor) I felt a need for more background to answer 
student questions* 

SA A N D SD 

Clarity refers to the concepts developed in the protocol as well 
as to the instructions to Field Test instructors on how to conduct the 
activities and tests. The following items exemplify the assessment of 
clarity: 

(Instructor) The objectives of the protocol unit were made 
clear to me by the instructor's guide. 

SA A N D SD 
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The differences In teaching strategies of the two teachers 
In the film were difficult to see. 

SA A N D SD 

Effectiveness as measured by the Evaluation Q^iestlonnalres refers 
to the perception on the part of the student and Instructor that the 
protocol materials were successful In achieving their purposes and did 
so better than other kinds of education courses vould be able to do. 
I have learned less from this protocol than fi om other 
activities In education courses. 

SA A N D SD 
(Instructor) I will use this protocol unit agai'n In my 
education courses. 

SA A N D SA 

Each of the six criteria (differentiating between "significance 
of concept" and "significance of activities") x^as assessed by a subset 
of items from the Student Evaluation Questionnaire and from the Instructor 
Evaluation Questionnaire. 

The items on the Evaluation Questionnaire were scored fronj one 

to five. A high score was indicative of a positive attitude toward the 

protocol unit. The average value of the subset of items which represented 

each criterion was determined for ease of interpretation. 

Illustration of three items which measure interest: 

1 . SA-5 A-A N*3 D«2 SD-1 

H- 4 5 6 5 4 (Scale Value « 3.30) 

^ Item Mean 3.00 
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2. SA>1 A-2 »-l P»A 
»• 2 S f 4 5 
ItM • 1.11 



1. Ufl A-2 »•) »^ 

IH 2 S 5 7 7 



fcaU «!iov* 3.0 for kMk aCMilMiCa Mi lMtrM«t*r« wf« cMi«M«r«4l 

«• MtlafMlory rulflllaviit of »Im rrlirrlM. 

TmC« mimimlmffd at (Im mtd •i •«cli prot«c«l Mi«rl«U Mft. Th« 

CrlurlM (••(■ ilfr«r#i In r«r« frMi frW«<t t* mmiUt^t , 

CMif«rmlat (• CH* MtlvltUs rrvacrlWvd fvr tb« r«r(&c«l«r uult. I#«|>.*w»*a 

•t«rr. A kith ^cMiCap* MCtfrtakU r»>>»» n » mmU U cmiaMmW 

Mt 1« CU •^•#W« Sf CAtvflll ^•lUlMCy CMC ««IU«(lMI. 



t. **!• tk* 0«l«»r«lC)r •! C«l*rai« rr«««r<tl n«t*rl«U «p?rMrlk 
Mires lAvtWr (Iw rr»l*«ttl rt^iwrlaU «^««rli imt* lf(tr«»i;«i^ 
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imimrmmtiwm thm f p r ia c l m la •clwr aiMailM c1mm«. U aMliic 

f <— f la tock KiM rui4 Tmi cI«mm Mii kIm Ci«>irt— tv««p ai 
tlw Wgiaalai aatf aai af tha liaa m'**' ^ ky tka flaU taai. 

Ito raafaaaaa af Kiw rUM Taat gr^f war* ataKlatUalljr c— »an< wlik 
»kM« af Kka Oaavarlaaa traar. 

tolas tit* ^•'tMK acara* aa a c*«arlaKa, • aM'fackar a««l)r«ia 
al c«varlaaca waa mmmd f caaif«r« Ik* ra*foiiM» •t thm ri«U Taai — 
tacUl tclaMa Naibai* traaf, tha riaU Taai -- CMMtal Nviliaia tr^^, 
sm4 tiM Caa^laaa frrni^, Tm akjacllvaa far aack af ika flva ^aJacis 
aa r rai aa tka 4afaii4a»K v«rlakl«a. 1W faaal-va^^r iMMal aatitra of 
tka 4aal«a (Caapkall md UmUf iaalfa 10) f€9<lmi9» tka paaalkllliy 
•f tfrairlag caaaal ewmUmimm fraa tka tfata* aiaca prr-KraatM«i 41f r«t«iK«t 
aaaat tka wtmf9 caaaaC ka r«ta4 aai. Maiiaver, ika mm af fra-iaai 
mtmmm aa MvarlMaa ■fitaa ika traa^ wltk taa y act la kI** acklava- 
■Mt praia t t ak)acKl«M. 

IW fallavUg iipiaiwat varlakiM vara a— 0—4 wlm ckU Oalcai 

•) l^aatlfjrlat, claaairjrli^, a««lyal«« varlaklaa ikai 

•ffaal l«ar«lat> 
k| laaaiAUlat Ik* l«|iarK«aca al eaa i apiaalUlai Ikaaa 

vwUklaa 

I. M^^ tfl ml Hi I r p iy ^t ar UtUa[ i fyltcatH a— far 

iMimiliiitLggUiifrfrtt^ 

• ) ■mfUlag mM Ma«Mllrlw la4Ui4<tal 41llaraM#« la 

4) ka««»aUlat ik* l««^llr«ilat«* •! 4llfaraa« la^riMr 

ckaraclrf lai Ic* far arraatlat la*iriMllaMl aliariMllvr*. 
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V»rMl IwffCtlon in tH» Conitlv Pi— .ion t The 
>»Utiwiiihi» Bat^*— 1> T«»cii«r ^mrM >«luivior ana Hud*«t 

•) ■ >c tK i«it 4itt9ttte9» Im tvaclMr varkal k«li«viort. 
f) t4Mtiryi«t MMlytiMt •td^Mit r««pen»«a is r«Ution 
f C««clMr •xpcctatioM*. 

Or«J«l«i— facta to Umck N—ni—fyl lalatiowhtpa 

t) MMCifjriat taachar MMvlara wtUch latficata taaching 

li) laca t aiaiag »tmi0M bakavlara tAicb iadieat* co«c«pc 
laaraiag. 

*• Fair VarM I ■^haviar 

1) biffara^attaciat balwaaa taaclirr varWal kahavtora lU»t 

•ra fall ar aafair. 
j) lacaeaisiM tiMt taacH^r wrkal kaliavior ia ralataJ to 

cagnltiva, affactiv* mm4 aocial atiaaiaca coHMinicatad 

ia tlia claaaroaa. 



friar ia Altar Bach Attar 

fiaU TaaC rrataral VaiC Fial^ Te^- 



WimU Teat 
Oraaf 


QaaatiaMi 
"A" 


aira 


Oritariaa Taatai 

•t«4a«t Kvalaailoa 
Qaaai iaaaairaa 


y 

QaaatlaAnacr 

"A* 


OasMTlaaa 
Oraar 


qaaatiaa» 
"A" 


aira 




Qaaatiaaaaifi^ 
"A" 






laairariar 
■iralaaliaa 
Qaaal iaaaaira 
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Ihm 4lMproltttMfit U timU ttstlnf cms iHim It vat tlr«^ 
coimt r^tyms. Om Social tcUnca Nathods and ona C«Mr«l iw. 
iMtmctor ftlaply rataniad tba ufM>peMd box of MtorUls. Only two 
SocUi SciOAco Natkoda Inaiructora ratyrood all of tka data. Thara vera 
partial raturaa froa Cka othar. All coaCrol Inatryctora raturnad 
backfroyad Mtariala aod pra-taat datat fcnir rontrol inaiructora 
ratyrn<4 poat-taat data. A tabic of raturna ia included in Appendix C. 
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D#«crlptl<m of the M«ult> 

In kMping with th9 porpo^ of this •valuatioi . 
findings of thm study vill preMtitod as they relate to the evaluative 
quaationa* organised by product. 

A. Mow af fcctiva is each product? 

1 • Conceptualising the Process of Instruction 

Thif; prodvict was deslfcned to provide preservice teacher* 
with opportunities to ob»^rve some mstructlcnal situationR 
and de^lop s conrc^ptual ratuevork :or aml^z^ing the instructlor- 
al proosaa* 



Criterion 

tigniflcancQ of Conce^it 

tignificance of Acti^fities 

Intevest 

Efficiency 

Clarity 

Bf factlv«meas (Perceived) 



scudent Instructors 
4.28 
3.11 

ji.03 2.64 
2.81* 2.82* 
3.4» 

n 2.8V 



* CritMion nut saticf act«rri]y achi 



The tadaical quality of thtt film wai; rat<:^^ #iBpecially low as 
vaa the ornaiaaation of the protocol unit. .Mi'iuu^h the instruct^^r i» 
Slight ba willing to use the unit in futuic* cour^, this protocoJ 
wss c(maider«J no sK>re effective than vha norualiv takes place in 



EKLC 



-13- 



•ducation cour0<30. The comments regarding weak features of 

the unit included the following: a) it was infiexlhle with 

regard to "one correct model"; b) it was too tme consuming; 

and c) it was a little unclear as to intent and dirmcted activities. 

Tiie criterion test was a descriptive analysis of a filmed 
instnictional sequence. Students were asked to lost and organize 
their observations. Hiis activity vas sinilar & actrivities done 
in the protocol unit. The test vas designed to wmre the 
ability of the students to describe and analyze mt-nd^^ 
instructional situation. Responses on five vmrz^-mm wrr scored 
by trained raters as either acceptable or unacc«rt«^*' . 
vmriaikles and the percentage of respondents Mtho^e re^ponseR 

wertr acem^cmble are as follows: 

a> Ability to distinguish and identify ttM^mtr t imtruc tional 

variables in a new instructional m±3ujbti^ MS. 
h) Ability to select out of the many dlst jJ^iim^mhV^ variables 

Im the situatioo those that have thi jmimm 1 il for 

relating to student learning — 92: 

c) Ability to aee the instructional ux;umti.ci* im dlftcrcnc 
dimensions and obacrve and identif> lables tlimt, taken 
together, wouiii give a complete des pM^mm etf MMt is 
going on in a given clsMroor — 45. 

d) Ability to enprrss observations in mmm crw ^ Iv^-amalytical 
language ratlu r than in the torm or njeem c n 65%. 

e) Ability to OTi^anlKC obeer vat ions ol f ^ enalytical 
statements concerning the relation^ .o^p: ^ivh^ Mei^hla. 



within a conceptual framework — 28Z. 
Aa a result of using this protocol unit* students were able 
to list their observations of an instructional situation in such 
a way that discrete variables arc sientioned. Furthermore » these 
variables are those %ihich are crucial to student learning. The 
obsftTvations are stated in objective rather than Judgemental terms. 
However I the observations are neither as complete nor as well 
organized as night be expected. 

2. Learners and Their Characteristics: Implications for 
loatructional Decision-Making 

In this multi-media uniti students were Introduced to the 
idea that learner antecedent conditions relate to the process 
of imatructioni acquainted with the types of data which are 
available to assess learner characteristics! and shown how 
Imatructional alternatives can be selected to match individual 
characteristics. 



Criterion Scale Mean 

Students Instructors 

Significance of Concept 3.39 4 .42 

Significance of Activities 3.01 3.58 

Intmrest 3.08 3.66 

Sufficiency 2.78* 2.90* 

Clarity ~ 3.60 

Effectiveness (Perceived) 3.00 3.16 



* Criterion not sat jR^octorily achieved 
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Coranents: 

Negative reactions centered around the third segment and 
the criterion test, which some found unclear in intent. Mention 
was made of the amount of material to be covered in a short time, 
as well as the contrived natmre of some activities. 

The instructions for the criterion test directed students to 
aMlyse a profile of infomation on a learner, to c^^cide frra 
»o g possible alternatives the best choice of objectives, rontent, 
strategy, materials, teacher role, nedla, and grouping arrsxi-^einent : 
and to jmstify their decisions. The response was scored sat.s-- 
factory if the decision was compatible with the lesrner's 
characteristics pmsented in the profile. Accordir? to the 
scores given by trained raters, CI percent of the siidll nurabcr of 
respondents (N*26) gave acceptable responses. The riajority of 
students completing this protocol unit were able to achieve the 
objectives. However* the achievement rate can not generalized 
to the portion of students who did not complete the criterion test. 

3. Verbal Infraction in the Cognitive Piry^ensior! T'.ie 
Relationship Between Teacher Verbal Behavior and Student Response 

In this protocol unit an analyt&c«l framework is provided 
to assist the student in recognizing variations in teacher 
verlMil behavior which influence the student response. Objectives 
for the students working with this protocol unit are the 
following: a) to recognise verbal interaction in the cognitive 
dinennion; b) to identify a unit of analysis useful lor 



interpreting relationships in verbal interaction; c) to 

make differentiations of teacher verbal behaviors based on 

a theoretical fraMaork provided; d) to differentiate between 

student responses that are productive from those which conforr: 

to the teacher's expectations; e) to hypothesize relatianships 

between teacher's wrbal bahavior and students* react Iobb; 

and f) to analyze teacher openness as a variable in interaction. 



Criterion 

Significance of Concept 

Significance of Activity 

Interest 

Sufficiency 

ClariXy 

Effacriveness (Perceived) 



Scale Mean 



Students 
3.71 
3.30 
3.06 
2.36* 
2.98* 
3.10 



Instr^tisrs 
A. 71 
3.72 
3.1A 
2.37* 
2.83* 
3.33 



* Criterion not satisfactorily achieved 



Comments: 

The quality of the sound track wmm the greatest weakness. The 
criterion test was viewed as confusing and a contrivance. The 
Jargon and technical langoage was a draw-back • aa were the awunt of 
material to be covered ami the lack af clarity af the directlans. 

The criterion test warn based on a filmed cimasroom discuaaion. 
The students were asked to classify the teachars* verbal .^havior 
and to predict the studamts* reapmmaea cm two instsKes aether 
verbal behavior. Only eighteen BtudwL& responded to this tamn 
aituatzlaai. Of thoaat 44 perc^ant gaiFe Kceptabla responses. F.f ty 
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students re^^rw n,' ' ^hr second part of the criterion test which 
was designed to measure th^ ability to hypothesize the relationship 
between teacher verbal bctevlor and student reaction. A8 percent 
of the respondents gave aneptable answers. 

Although the student^-^er elved and Instructor-perceived effective- 
ness was adequate as memmmmd by the evaluation questionnaires, the 
responses to the criterion t**fits were less than adequate. However, 
there is evidence that It ns the criterion test itself which 
failed, rather t'.an the prxwtocol materials unit itself. The 
limited ru >cr of students who respooded attests to this latter 
Interprrta: Lon. T \c instznctlons were unclear and the reader can 
only guess what the test aevelopers meant or expected. 

As mentioned above, ciie clarity criterion was not met, largely 
due to the criterion test itself. Furthermore, the achievement 
rate cannot he genzralized to the portion of students who did not 
complete the criterion tsar. 

4. Organizing Facts o Teach Meaningful Relationships 

The purpose of thlm protocol materials unit was to assist 
preservlce teachers 2m identifying the essential attributes of 
concept teaching and riM teacher verbal behaviors inherent in 
concept teaching. In a film shoifimg two classroom eplseAes, 
the 8tud«its have a clMmcie to receptize and compare instacnces 
and no^-instances of ommccpt teacMmg. 
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Criterion 



Scale Mean 



Students 



Instructors 



Significance of Concept 
Significance of Activities 



3.96 



3,31 



4.60 



3.56 



Interest 



3.37 



3. 30 



Sufficiency 
Clarity 

Effectiveness (Perceived) 



3,37 



3.22 



3.96 



2.70* 



2.90* 



3.71 



* Criterion not satisfactorily achieved 
Comments: 

The criterion test is a transcript of a teaching sequence In 
which the students were asked to indicate whether the characteristics 
of concept teaching vere being used or violated by the teacher. Of 
the small number (N«21) responding, half were able to give an 
acceptable number of characteristics. About half of the respondents 
were able to identify learner responses which are indjcatlve of 
concept learning. Half of the respondents were able to recognize 
that the teacher verbal behavior was an instance or a non^lnstance 
of concept teaching. The effectiveness criterion for this unit 
was not satisfactorily attained by the proportion responding. Again, 
the results cannot be generalized to the population of field test 
students. 



5. Fair Verbal Behavior 

This protocol unit was designed to help preservlce teachers 
differentiate between teacher verbal behaviors that are fair and 
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those that are not fair, to iJentify specific teacher 
behavior patterns associated with fairness, and to recognize 
the implications of fairness and non-fairness in the pattern 
of classroom interaction. Color filmed segments of teacher 
behavior and student response compose the stimulus. Students 
work with transcripts to idaatify instances of fair and unfair 
verbal behavior. They also role play a fair and an unfair 
teacher reaction to a studem: response. 

Criterion Scale Mean 



Students Instructors 

Significance of Concept 4.07 3,28 

Significance of Activities 3.35 3.33 

Interest 3.85 3.66 

Sufficiency 2.80* 3.07 

Clarity — 2.78* 

Effectiveness (Perceived) 3.49 3.28 



''^ Criterion not satisfactorily achieved 
Comments : 

Written comments on weak aspects of this unit included: 
a) the scholary orientation; b) technical jargon; c) contrived 
filmed situations; and d) a redundnt, useless criterion test with 
unclear instructions for Its use. 

The criterion test was composed of role-played responses to a 
series of student comments. One nspomic was to bo a fair response, 
another an unfair verbal response, of the thirty-three students 
whoa>mpletcd this test, 9A percent gave saaux^f actory responses. 
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indicating that this part of the protocol unit objectives had 
been net. 

Sunanary 

For all five protocol products, it can be said that both the 
ideas dealt with and the activities used to deal with them were 
considered worthwhile and significant by both students and instructors 
The interest level of the protocol units was good with the exception 
that instructors did not consider Conceptualizing the Process of 
Instruction to be of interest to the students. Learners ard Their 
Characteristics; Implications for Instructional Decision-Making , 
Verbal Interaction in the Cognitive Dimension; The Relationship 
Between Teacher Verbal Behavior and Stud ent^es£onse , and Fair 
Verbal Behavior were perceived as effective by both students and 
instructors, while the latter group did not consider Conceptualizing 
the Process of Instruction and Organizing Facts to Teach Meaningful 
Relationships to be as effective as alternative approaches to teacher 
education. Fair Verbal Behavior and Verbal Interaction in the 
Cognitive Dimension; The Relationship Between Teacher Verbal Behavior 
and Student Response suffered from a lack of clarity. Sufficiency 
was the criterion which was not met most often, usually as a result 
of the technical quality of the audio-visual components. Considering 
effectiveness in terms of meeting product objectives, few general 
comments can be made, primarily due to the response rate, but also 
due to the nature of the criterion tests. 
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B. Is the University of Colorado Protocol Materials Development 

Project approach different from conventional approaciies to teacher 

education in terms of outcomes? 

Questionnaire "A'*, entitled ''Concepts in Teaching/' was 

administered to the field test students and comparison group 

students before and 4»fter the field test period. Subsets of this 

questionnaire (groups of three or four items) related to different 

product objectives. A one-^factor analysis of covariance> with 

pre-test as covariate, was used to assess the differences in means 

of three groups: 1) Field Test — Social Science Methods Classes; 

2) Field Test — General Methods classes; and 3) Comparison groups. 

The results of this analysis are presented below for each of the 

products. The statistical tables are presented in Appendix D. 

1. Conceptualizing the Process of Instruction 

eO"Identifying, classifying » analyzing the variables that 

affect learning." Although the pre-test significantly reduced 

the unexplained variance, there were no significant mean 

differences among the groups. 

b) "Recognizing the importance of conceptualizing these 

var iables . " There were significant adjusted mean differences 

among the groups; however » the differences were between Social 

Science Methods and Comparison Croup on the one hand, and General 

Methods on the other I 

Adjusted Mean for Social Science Methods » 11.93 

Adjusted Mean for Comparison Croup « 12.23 

Adjusted Mean for General Methods « 11.26 

EKLC 



2 . icurnTt m4 TUvlr CU, i r^.uj rrUtlcii Iwlica tlon* for 
Inetruc t lo nai Dccltilon-M.il s in|; 

c) '*Kri'o;:nlilnt and IJiMWHylnK lndlvldii/>l ilffcrvncrs In 
tttltudra toward nchool iK Npltc thr •Ignlf leant contribution 
of th« pro-t««t uMd mm m covariata. thara %mrm no algnificant 
poat-traataant diffarancaa in tha adjuatad Mana, 

d) **RacotniBlnK tha Inpllcationa of dlffafant laarnar 
charactariatica in arranging inalructlonal altarnat Iva^ 

On thia dapandant variabla thara %#ara raliabla diffarancaa in 

tha adjuatad Mans of tha group. In thia caaa, Canaral Mathoda 

and Coaipariaon Qroup aaana vera approximataly aquivalaot, vith 

tha Man for Social Sciaaca Mrthoda indicating ralatlvaly laaa 

achirvnicnt on thia variable. 

Adjuatad Maan for Social Sclanca Hothoda - 9.67 
Adjuatad Haan for Cmaral Mathoda • 10.^4 

Adjuatad Hcan for Cospariaon Croup • 10.89 

3 • Vi^bal Inte raction In the Cuynitlva Diaanaioni The Halation ahlp 

Satiieen Teacher Varbal Behavior and Studant ILmmfotkmrn 

c) **tocotnialnf: differencea in taachar varbal behavior.** 

Tha adjuatad Man tor Social SciaAca Matilda vaa aignificMtly 

higher on thia varl.ibla than vara tboaa of tha other tvo groupa. 

Adjuatad M« ..n for Social SciaAca Mathoda - 12.9) 
Adjuatad Mf ui for CMorai Matilda - ll.M 

Adjuatad M« m for Cfi Bpariaoo CroMp - 11.41 

f) ^'l^antiryln/. and aMiyuiac atndant raaponaea la ralatloa 

to taacliar ajc^tat f 'hiii.** The adjuatad aiaaa for tha Social 

Scioacc Ifc thoda r.r" M* vaa algnif Icaatly greater thaa tboaa for 
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th« Cmnmrml Methods or tho Comparison groups. 

Adjusted MsAii for Social Sclsncs Methods • 12.62 
AdJusCod Hssii for Csnsrsl Hrthods - 9.96 

Adjusted Nssn for Coiipsrlson Croup • 9.^9 

4 . Org/tnislnt fscls to Tssrh M rsnlnsful Ks]atlonthlpi> 

t) ^'Idontifying tssclisi behaviors which indicato tsachlng 

r^r concspt attaliwont." Ihs adJust^Kl *Man for ths Social 

Sclrncs Mrrhods group was significantly greater than tho5« for 

ths Gansral Hathods group and ths Comparison groups. 

Adjustod Itoan for Social Scisncs Methods • 11.37 
Adjustad Msan for General Methods • 9.S6 

Adjusted Mean for Comparison Croup • 9.59 

b) **necogniting student behaviors vliich Indicate teaching 

for coMept attainmnt.'* :«o reliable differences vsre found 

aaont the groups on this variable. 

* • r*ir Verbal leh^vlor 

i) **Dirrcrontiating between fair and unfair teacher verbal 
behaviors.** Hiere were slgnlficnat differences atx>ng the means 
of rhe three groups. lUmever. the siean for the Corrparison group 
^mm sigAif icently greater rhan those for either of the field 
test groups. 

Adjusted Mean for Social Science Methods - 11.33 
Adjueted Mean for CMeral Mettiods - 11.32 

Adjusted Mean for Comparison Croup - 12.77 

j) **lecogmi8img that teacher verbal behavior Is related 

to cogmitive, affective, and social moanings cooaunlcated in 

the classroom. Tlii^rs Mere do significant diffsrencss a^zou^. 
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thm Mans of th« thv groups on this variable. 

Takan togathcr, tho raaulta from Quaatlonnaire "A", given 
to Plaid Taat and Compii'^iaon groups, Indicated that those students 
who used the protocol Mterials were no more likely to fare vei: 
on these measures than were students in conventional educational 
classes. Within the limitation of the Questionnaire for measuring 
these variables, the use of the protocol materials did not differ- 
entiate the testing population, that is. those who had used them 
from those who had not. This finding interacts with the differences 
found b«t%reen Social Science Methods and General Methods, however. 
Perhaps the focus on social science classes in the protocol films 
provided an advantage to the Social Science Methods group, although 
even this is not consistent from variable to variable. Furthermore, 
an inspection of the means of the different classes themselves 
suggests that there may have been vithin-group. between-class 
differences. This implies that the instructor characteristics, 
student entry behavior, etc. should be considered in setting up 
future field taste and that the unit of analysis should be the 
clasoroom. 

Although the above analysis deals with differences in outcomes 
batveen the protocol approach and conventional teacher t <n 
methods, xtems on the Student and Instructor £valuati . gi nnaires 
dealt with differences In process . These wre assessed by items 
such as the following: "Compared to what normally takes place in 
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my education courses » this protocol \it was norc int 

Protocol Scale Mje 

Students 

Conceptual jglnf} the Process 

of Instruction 2.82 

Learners and Their Characteristics 2.92 

Verbal Interaction in the 

CoRnitive Dimension 3.04 

Orfi , aniglng Facts to Teach 

Meaningful Relationships 3.19 

Fair Verbal Behavior 3 . 38 

Both students and Instructors apparently perceived - :he 
processes of Verbal Interaction in the Cognitive T^iiMmm^ Jie 
Relationship Between Teacher Verbal Behavior and StMimt x>nse 

and Fair Verbal Behavior taught them more than what tte^y * d 
ordinarily learn in a teacher education course. In . hut 

not students » thought that Learners and Their Chara> s:^ 

Implications for Instructional Decision-Making offer* ^ oetter 

approach than the conventional, while students felt tha way about 
Organizing Facts to Teach Meaningful Relationships > 



Judgements of the Results 

Judgements are a necessary part of evaluation. Different perspectives 
on an educational product are held by consumers, developers, 'He 
evaluators, all of whom can Interpret the evaluation data x ent 
ways. Each can find an answer to the evaluative questions from the data 
gathered . 



A. The Bm cor ^ Viev 

Looking *c * ocol -^^oducts md l. ^ ^hcred by the 

development st npresfl^^ mainly bv s. The first 

impressive aspec e tremendous amc ' *f that has gone :nt? 

the five units at we am the qualxry o :^«'9^ lization that imk 
have gone into the ? Txing ana developi:! :r i^roduct. SecoKi. 

I am impressed wjn commitment on t: .-ir- i staff to seei 

empiricci 1 answers o developmental questio ^ commitment is 
larking in too mmm\ educational endeavor:; u em irical answers, 
however 9 were no* as straight-iorvard anc «n»*qu . 1 as I would have 
liked. Indeed* ^'^THrtrher the products were ^t^<er a • <dr test or not may 
be in doubt. Tie c'^velopmental process sh *»\- - tin, between prototy 

development and Id test, an intermedials si» oi :^ilot test, in 
which a potential consumer can go over ea N 5 x uader the wat«*hful cy^ 
of the developer. nxs step should be li^f^u : rify ambiguities in 

instructions, etc out most of all to evaxuarra hr instruments, assess 
their reliability ontent validity at lt*ast. a«Ml 'heir appropriateness 
prior to the fieln test. If the Protocol staf: been able to take 

this step, several voblcms \tomid have been ave^uec These problems 
centered around claraty of insiractions and testini expectations to 
extent, but most importantly er^ the tests themse The failure of 

more than half oi Hsr students tr take the cr i i u tests speaks to 
this poinr. £ell»»re to «ci4i«re the ef fectisim -g?^ crl'^rion (for 

some prodvct^* cotUd iMve had m^h to ^ ' tfav t' t« as a with tlie 
products themselves. he sant aint is tr^e rr }m^^*^ imnaire *'A'*, 
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**Conc^pt8 In Teaching/* vhlcn was never pre-testet mmd e > lated. 

Certain other aspects ^^f the evaluation smml vr t iltlvely. 
)ne is the worth of the concepts and Instructioul acr- vrii» percelvec 
for ^1 products by both instructors and studentx uKd v h ^ . level 
of interest generated. It lb also noticeable tbmc tlt^ chr\ .:al qual ~ 
of tht media components was often less than adetfrnrnxm^ ^vel of 

Ideas and language may have been too much for the iiei^c^ instructors 
to grasp. 

Despite the weaknesses mentioned » my own ivmr^umi^ ?n the 
Protocol Project — both processes and products — Is ve positive. 
It Is a worthwhile contribution to teacher eciucatiio^ 

B. l*he Developer's View 

The data gathered In the field tests indlcatw tt -im rikat the 
concepts we selected are worthy of development as ^nfucaJ i^erlals. 
The attempt to ''conceptualize the process of InstrvmoK** m> be an 
exception, and I would use the idea for an overvlw ^ criterion 

test rather than as a protocol. In any case, the ept^odeMi have to 

be redone with vastly Improved sound. 

Tlic field test gave clear direction lor rwirnxmu a flpcclflc 
products: 

Learners and Their Characteristics: Ixapi^ i r itggB f iii 
I— tructlonal Decision-Making needs a coapletelf 41ft»rMt 
criterion test and 1cm cMitriirttd clanur^ i «r^rirtaB. The 
cMAette-tapt' which accciiai'ilns Part III, ''Insiructianai Altermmtitv^^ 
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•gg d c to kr rerecorded for a mere natural effects and the entire 
wmit ^mmmm to be shortened. 

^^rbjia Interaction In the Cognitive Dimenston; The Relationship 
iMr— gc ^kfcachcr Verbal Behavior and Student Response needs a new 
crlt^^rijB Lcst and the film will have to be reworked to improve 
the o i un i in some epic^odes. 

ip ganir:ing Facts to Teach Meaningful Relationships requires 
clar. icatiion in Instructions in the guide. 

ffe. ir Verbal Behavior needs a revised and simplified teacher's 
guick: and a new criterion test. 

All of tke criterion tests, when reviaed, need to be pre-tested 
for reliability and validity. 

The data suggests that the Protocol Materials approach is more 
interesting to students than conventional approaches and that Instructors 
will OK sue materials (as revised) In planning future courses. It 
appears thsi the Colorado materials are more appropriate for Social 
Studies Metibods classes than for General Methods classes because of the 
content of the classroom episodes and because the process dimension of 
protocols is familiar to Social Science Methods teachers. 

Ike revisions outlined abovc» including pre-testing of instruments 
and rsMking one film» can be accomplished in 4-6 months. The 
aatlmatni oast would be $15^000. 00 » including salaries and materials. 

I mmad to aiic that our expectations for the field test were not 
«et* primarily due to the pressure of time. We felt we had an obligation 
to test tir proAocts in the spring of 1972 so that the products would 



be ready fc- oae or revision in September 1972 The field test 
instructors :iad no time to preview the pre uct-£ bofor- using ther : 
class; i*e hx no aoney to provide tor an inservice workshop for iT:.iiru, 
and they had to depend on the guides for direction as to use. Man 
instructors simply did not give the criterion tests. Others ran on 
of time to try all five units. 

We have reported the data as we have it. More complete return^ 
might have given better information on the two evaluative questions we 
asked . 

C. Tha Consumer's View 

As a prospective consumer, I would consider very sericMsly using 
these Protocol materials. In support of ttiis judgement, I axfer the 
following comments. 

The use of imaginative films and other media in a mctij^is course 
is attractive to me. The data [gathered f--om this evaluation does indi t 
that one of the strong features of the ■■terial is its perceived intert- v 
and worthvhilencss by both students and liMchers. I also am iinpre5%sec 
by the effort that went into evaluating rir^jse wterials. ALtiumgfc the- 
evaluation is far from unequivocally sup^oc^t^sve, it helps the user to 
Identify trotiblesomQ aspects of the materials. 

An area of concern, according to this evaluation, is the ciaaparisr-r 
of these materials to traditional claasrvH j^tho4%. When one iav^sts in 
a clasarooM innovation, t is necessary to realize a coffviensurate gain 
in the instruction of that classroom. Tl»ere is little miication that 
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the i»r.uaents j he experlwe :al group learne^ iwr jn the criteria 
sel€K*t^ thai' t ,ose in the :c-^trol grou: . Th. . cw-ver, is not dis- 
cour . xig f*- ar least a coi- of reasons, ^zrr . the subjects iv^ 
a di:.. lian- -nirr vjith the cr^zerion te- tliat we-t administered. In 
additiu a rt .^Lively small uumber of i>ubjec s cc^pleted the tests, 
reliabj it anc validity of the instruments is ^uist, if not more, 
suspect whan tht films. Further analysxs of : he v^i^sVf^ is needed. Otlu 
short coBii ngs in t he material was its suf f icieru:y ir ar ;i«'ving their 
Instr icti^nal ejals. Given the diraeiuKLons or learr..^ co\'ered by the 
material, it 1"= unreasonable toaispect sufficxent cxiverage. These 
materials, ii. rrv judgement, ought to be used t« susj^jleaent other mate: ^ a:, 
and methcxis ir course rather than suh=?tititt«' for them. 

Overall hese matarials received a ^i^tmrl rat xng fro«i bot^ -tua'-^ts. 
anii teachers. ii using sociil science ire^tkcMlic c' asses for the co-rr_^ 
fif'Ouj I he de iners of this Gvaluati^r^ ..Aad. it «yt ^re fair, t 
am^BSr'Gdy bu '^c rigorous. Persoc^J 3 1 ^fcmlL ts^^ the «atu±:r , j : 

4«nc ril metkecis lass raaiher thai, a n:ia! ^xeice methods lass.. Tne 
Q^ecti.vcs of he materials are aiskd more a. r.jriirhnr tradnij^ 
mmd thr content is not necessarily social m mmm i iiAJBai^ 

vS ^ potential -^nsiaa&er , some seriom^ iue«K:.lflK do C4iaie to mine 
was tMK ioned witi fj* v«*al of the units t-ttar tir*t "^educational jargcrr 
mmd zim te:nnica2 qu*.: ty oi the 1 il»f{ were <dxmtr^tJkm^ to the UH^rs. 

' I *fi.^ild H# more •coiratc to - ly cJhan T m a prospective 

trial i^elt, caster »f iIk: w^fterial ratiiM tlus £ final commor. It 

soumiw as ii i.M ]Yot»rrL»£ *'aduct hu^ ^oteS d ^ of potcacial t.L _s 
^ -wfrision amyr froir ti^ market. 
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APPENDIX A 

List of Field Test and Comparison Group Instructors 
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ADDRESS LIST 
FIELD TEST 1NSTRUCTCM5 



Social Science Methods Experimental 

Lichtenberger , Edgar (55 students) 
School of Education 
University of Texas at El Paso 
El Paso, Texas 79968 
(915) 584-7130 

Mackey* Janes (25 students) 
School of Education 
Peik Hall 152A 
University of Minnesota 
Minneapolis. Minnesota 
( ) 373-9721 

Pratt, Robert (20 students) 
University of Northern Iowa 
Cedar Falls, Iowa 
( ) 273-2362 

Schieffer, Joseph (35 students) 
San Fernando Valley State 
18111 Nordhoff 
Northridge, Calif. 91324 
(213) 885-2581 

Smith, Sallie W. (20 students) 
Teachers College, Columbia University 
525 W. 120 Street 
P.O. Box 222 

New York, New York 10027 
(212) 870-4257 

General Methods Hxperimental 

Burnham, Glen (40 students) 
Division of Education 
Adams State College 
Alamosa, Colorado 81101 
589-7936 

Carpenter, Ruth (25 students) 
Florida State University 
916 West Park Avenue 
Tallahass-e, Florida 32306 

Farrer, Kenneth C. (20 students) 

School of Education 

Education Building 104 

Utah State University 

Logan, Utah 84321 

( ) 752-4100, Ijct, 7385 
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Odioa, Tony (20 students) 
Mexican Aiacrican Studies 
C«llf. State College at Hayward 
Z3«D0 Hillary 
Hsyvard, Calif. 94542 
(415) 884-3263 

Tack, Marionette (30 students) 

Dei^rtnieiit of Secondary Education 

San Jose State College 

125 South 7th 

San Jose, Calif. 95114 

(408) 277-^2642 

Social Science Methods Control 

Cl«:gg, Ambrose, Jr. (45 students) 
120 Miller Hall 
University of Washington 
Seattle, Washington 98195 
(2m) 543-6636 

Dav^&s, Daniel F. (25 students) 
Ohim State University 
1943 North High Street 
Colozabus, Ohio 43210 
(6U) 422-1080 

Kirby, Darrell (35 students) 
P.O. Box 3 AC 

New Mexico State University 
Las Cruces, 2^ew Mexico 88001 
(505) 646-1429 

Kleg, Milton (20 students) 
CoUegr^. of Education 
University of South Florida 
Taapa, Florida 33620 

Richburg, Robert (30 students) 
Colorado State University 
Fort Collins, Colorado 
(303) 491-6009 
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APPENDIX B 
Instruments 
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Protocol Materials Development Project 
University of Colorado 

INSTRUCTOR BACKGROUiNT} INKORIIATION 
1. NAME HOME ADDRESS 



ERIC 



COLLEGE OR 
UNIVERSITY 



ADDRESS HOME PIIONU 



OFFICE PHONE 



2. Approximate number of semester hours you have taken in: 

Social Sciences (History, Anthropology, Sociology, etc.) 

Educational Psychology 

Curriculum and Instruction 

Other area(s) of emphasis in Education (Please specify) 



3. How nany years of teaching experience do you have at the college level? 

A. How many years of teaching experience do you have at the .secondary or 
elementary level? Secondary LUenentary 

5. VTliat was your undergraduate luajor? _ 

6. Major for Mastc^rs Degree? ^ 



?• >lajor for Doctorate Degree? 



8. How many students are in the clcr»s you are using to try out the laaterials? 



9. Wiat dalles will you be using the materials for the field test? 

From to^ . 

What days and times does the class meet? 



30. What texts or otlier published iiuiterirls, if any, v.'ould you use in the pre- 
service methods course you nornially teach? (Ansv/er on back of sheet) 

11. What major topics would you cover in 1)k» jue-servicc course you would • 
normally teach? What is the relalive c:;\ph:>sis on each of tlior:e topics? 
(Answer first part on back of sheet) 

TOPIC DEGREr. OF ElIPHASIS 



Protocol Katcrial^ IK^velo|«««nt Project 

University o( Colorado «^ 

- Jt#- 

tRsnturrcm r\'ALU.\VK% gcLSTto:«7;Aiitc 

tot 

CMic^rlualisine the Process of InsCrwcCloo 



iMCructor^s Kaw^ ft:;lo 



1. Nov WKh class cIm In rlmiCcs 414 jrou spond oti this proCcKcl 
Mil? 

2. DoscrlW ch^ ^cvrcdure yoy Mood in frcsmclnr this protocol 
SMiCrrlals unit. Did yuu use the subtest ions In the |tut<!cT 
Any vsristloos? 



). WtwiC Jo yo*i ChtiiV \ Ik iKi tsln thtiif yvHir ^CuJrntr. It.^ifUit 
(vota this ptutfcol uiMt? 



ItvdIcAtr your Jf:;rc«t of arrtr-cnt %li*t rrcb t»Csl rtj<«fii. KsrI ypur ttr; . c-i^ 
in itie ^r^ACo prrvldod isforr racli r^tc ifo.i. 

lA - 5^troiirly sfrrr A • Afrro R » Ko opiN|«tn 0 • Uimu$ji0 
VD • Mrot<:ly ditisrroo 

A. Ills concepts dr^lt vUh iu prolpccil wJt sit- v\>r tti«/lif li for 

tcocWtr priT^istlmi. 

^.^^ CcK^^rrJ ti> vlt^r n€it illy IaWs pl^cr In C'Jurstlc'n le^tsin^ itir 

proloiol unit w4iL lilt t 1 1 ^>t iar * 

6. The siiMft^nts lr*irn«-.* lt»^ frot tMs piviocol unit tti^ fron rt*rr 

«€tivlll<i^ In rducMlun rcKit'»#s. 

I. Thr studtfttn fm«t»t llie ft In Worin^. 

, t. Thr pirtMrr ^iillty of Chr fllii nrrds I* prcifni*t>t 

Thr Audio ^sllly of Chr fllu n<-«4s ii.prc^t ««mt . 

10. Ns¥iii/ rrtidi^Ct^ ll^t fH^*li ohs4*rv^t lof»r trim tU (frs. t^w «|i;<s}ir 
of th«* ffUi MS Mrilvwl<ll*. 

, ll« IA»rlli*n Is t4Mll grMups to rl'«o»lfy tW« It mI ^^^^rv^t % •<» # ^<«fil- 
i4lll« activity fni mudrntik. 



CMc^ftMlUittg the Process of Instruct ion (coo^t.) 

12. tevlng transcripts of ths cl/issroo« splsodos in ths fila for 
roforoncs ims vsrjr hslpful. 

i 3> Roving studonts drsv s ttodcl of the process of instruction vasn*t 
0 vory vorrlivliils activity. 

14. Vritin^ s dsscriptlvs analysis of ths last episode In the film 

Mia a vorthwhllo activity for students. 

I >« Ths content of the three classroov episodes in the film needs to 
ho MTo varied. 

1$. It hothered sie that all three episodes in tlie f ilsi depicted social 
studies classroras. 

17. The sufjtestlotts for teachlnc procedures are clearly doHcribitf in 
the e^fde. 

II. 1 felt a need for mnrm hackground to anr.Wvr student quest loiif.* 

^ 19. This protocol unit n4i*ds to bo better or^anlxed. 

JO. Ths otjeetlveft of the protocol unit vers made clear to me by the 

iMtriiCtor'a guide. 

71 . t will uae this protocol unit agnln in my education courses. 

H. This protocol unit needs 9.1 jor rcvlsionf; before furtU^r use. 

23. In the space below discuss the weakest frsture of thlh protocol 
MUlt and what can be done to loprove it. 



»4. 



Aay oth«»r cosMentsT 
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IMSTRUCTOR EVALUATION Q tesTION-NAIRE 
for 

Learners and Their Characteristics: 
Ittplications for Instructional Decision-Making 

Instructor's Nane Date 



1. Hov much class tine in minutes did you spend on this protocol 
unitT 

2. Describe the procedure you used in presenting this protocol 
materials unit. Did you use the suggestions in the guide? 
Any variations? 



3. What do you think vns the nain thing your ctudonts learned 
froa this protocol unit? 



Indicate your degree of agrecncnt villi each statement. Mark your responses 
in ilie space provided before each qxirction. 

8A » Strongly agree A » Agree N • Ko opinion D « Disagree 
SD * Strongly dir.agree 

4. The concepts dealt vith in this protocol unit arc worthwhile for 

taachrr preparation. 

5. Compared to what normally tf»kec piece in education courses, this 

protocol unit v^as more intcrcnting. 

6. Tho ittudents learned less frcni this protocol unit thnn from other 

activities in education courses. 

7. The students found the slides on "Student Attitudes Toward School" 

Intcrcstiug. 

^ 8. Tlic quality of the slides on **Student Attitudes Toward School" need 
inprovement . 

9. Tlie slides on '•rtudcnt AttitudeH Towjrd School** stimulated a 6i[;nl- 

f leant dlscucf^iMn. 

10. Working vitli llic- student profiler vras vurthwhile for my stncUMiis. 

11. The i;tui!riitn felt vorking vith the stiuWnt prof lit s was borlii^;. 



2 

Learners and Their Characteristics (con*tO 

^12. The quality of the slides on "Instructional Alternatives" needs 

Ijaprovement • 

^13. The audio quality of the cassette tape accompanying the slides on 

"Instructional Alternatives" needs improvement. 

^14. The slide tape on "Instructional Alternatives" wasn't very infor- 
mative for my students* 

^15. The students felt the slide-tape on "Instructional Alternatives" 

vas interesting. 

^16. Working with the student profiles following the slide-tape on 

"Instructional Alternatives" was worthwhile. 

17. The suggestions for teaching procedures are clearly described in 

the guide. 

^18. I felt a need for more background in order to answer student 

questions. 

19. This protocol unit needs to be better organized. 

20. The objectives of the protocol unit \;erc made clear to nc by the 

instructor's guide. 

21. I will use this protocol unit again in ny education courses. 

22. This protocol unit needs rajor revisions before further use. 

23. In the space below discuss the weakest feature of this protocol 
unit and what can be done to inprove it. 



24. 



Any other comments? 
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INSTRUCTOR EVAI,UATION QUESTIONNAIRK 
for 

Verbal Interaction in the Cognitive Dimension: 
The Relationship between Teacher Verbal Behavior 
and Student Response 

Instructor * s Name Date 



1« How much class time in minutes did you spend on this protocol 
unit? 

2. Describe the procedures you used in presenting this protocol 
materials unit. Did you use the suggestions in the guide? 
Any variations? 



3. VHiat do you think was the main thin^; your students learned 
from this protocol unit? 



Indicate your denree of agreement v;ith each sLntement. Hark your responses 
in the space provided before eacli question. 

SA « Strongly agree A » Agree N - No opinion D » Disagree 
SD " Strongly disagree 

A. The concepts dcclt with in this protocol are worthv:hlle for 

teacher preparation. 

, 5. Compared to v;hat normally taken pJaue in education courses, this 

protocol unit was more interesting. 

6. The studentc learned letis from thie protocol unit than f ro;a other 

activities in education courr;es. 

?• The students found the film boring. 

8. The picture quality of the film needs improveiiient . 

9. The audio quality of the film needs improvement. 

^10. The discussion of the anolyticnl fraiacwork developed by Mncdonald 

and Zarct was'o't very significant. 



2 



11. Watching the first two episodes ot the fxzm marking the 
Matrices was a worthwhile activity for stadesir . . 

12. The overprint that described the interact lor in the classroom 
con f u sed s tiid en 1 8 • 

13* The purpose of predicting the responses in episode three wasn't 
clear to me. 

^14. Having transcripts of the classroom episodes in the film for 
reference was very helpful. 

^15. This unit was teaching the Macdonald-Zaret framework, rather 
than focusing on one way to look at verbal interaction. 

^16. The throe episodes of the film illustrated tlie cognitive dimen- 
sion of verbal interaction. 

17. The suggestions for teaching procedures ar^ c^Mrly im m r i bed 
in the guide. 

18. I felt a need for more background to ansvycr iradcnt qo-^^ions, 

^19. This protocol unit needs to be )ettcr org*:^. -ixrd . 

^20. The objectives of the protocol unit were nim* clear to me by the 
instructor's guide. 

_^21. I ' ll use this protocol unit aj^iin in my ♦ttocj t i on cofirscs, 

22. This protocol unit neede injor revision bc:w farther v%c, 

23. In the cpnce belo\' dlscutis the weakest feature this protocol 
unit and what can be done to improve it. 



24. Any other comments? 



EKLC 
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iNsnccroR evaluation questionnaire 

for 

Organlxing Facta to Teach Meaningful Relatlonahlpa 



Instructor* a Waa a j^^^ 

1. Bov mch class tine In nlnutea did you apend on this protocol 

ImltT 



2. Describe the procedures you used In presenting this protocol 
Mterlals unit. Did you uae the suggestions in the guide? 
Any vsrlotlons? 



3. What do you think was the main thlitg your students learned 
frott this protocol unit? 



Zndlcate your degree of agreement with each ^tatciseut. Mark your respoaccs 
In the space provided before each question. 

fiA • Strongly aereo A - Agree H • No opinion D ■ Disagree 
fD • Strongly disagree 

conccpta dealt with In thla protocol arc worthwhile for tcachcr 

preparatlon. 

5. Cottpared to what noraally takes place In education courses, this 
protocol unit waa tooro Interesting. 

Th« students lesrnod less from this protocol unit than (xm other 

actlvltli!s In education courses. 

The students found the film boring. 

The picture qusllty of the film needs Improve.acnt . 

9* Ths sudlo quality of the film needs improvement. 

10> The lesson vas too conipllcated to tench. 

The w>rk or definitions of relstad terms wss necessary. 

The lists of verbal iudicstors of concept teaching wore helpful. 



13. The film should have been ehown earlier In the unit* 

14 • Analysing the transcripts of the two cplsodec vas vorthvhlle* 

15* Bavlvg students teske a list of non-exasiplcs of verbal Indicators 
of copcept teaching ws worthwhile « 

16* The differences In teaching strategies of the two teachers In the 
fllM were difficult to see* 

17 • The wggestloas for teaching pr«-cedures are clearly described in 
the guide* 

18* I felt a need for nore backgroiad to answer student questions* 

19* Thtm protocol unit needs to be hatter organised* 

20- objectives of the protocol isalt were cade clear to me by the 

tetructor*a pside* 

21* I will use this protocol oslt again In my educatlcc courses* 

22# Tbia protocol nit needs maiar revision before further uac. 

23* In the space below dlscusr the smikest feature of *hlfi pr zvl . 
unit and what can be done te i&prove It. 



24* Any o^cr coanscntsT 
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INSWLXUR EVALUATION QaESTIOnMlRE 
for 

Fair Verbal Behavior 



Iiifliruct«r*a 

I. 



Date 



i much clwe timm la m 

Lt? 



did you 



on tM« precocol 



rrlke tiK firocedures 
MteriaU unit. Did ym 
Any ivtetlcma? 



protocol 
guMe? 



3. Vhat » ^w think vkb tiH 
fron t - pmtocol tMlt? 



chlen; your atudt its learned 



Indicate ymnr degree of asr ii mm with each Matwmt. Mark yoiir rraponses 
In the epnce prorlded wmimm each queatimi. 

SA - Stroncly agree A - 
SD ■ StroniiT diaagree 



5- 



^ 7- 

_ «• 
^ 9. 
^10. 

11. 



12. 




N - Ko 
with in thlG 



D • Diaagree 

are vorthwhlle for teecber 

to idwt nonially takes place in education ceuraes^ this 
protnael unit una more internet ing« 

The atndente learned less fren this protocol unit than from other 
activities la education courses. 

The students found the film boring. 

The picture ^ality of the filu neede lAprovcMnt. 

The audio quality of the fllai needs jjnpiiiu tiiienl . 

Thla color film wns better and more interesting than the same film 



wonld hove 



in black and white. 



asked during the film didn't seen to s^-rve any useful 
f cless. 



The quest! 

purpose for 

The behavior of the first teacher in the film uaa tor overdone to 
be believable. 



dlscuEslon following the film wasn't very significant. 

^^^^ ^^^ transcripts of the classroon episodes in the fite in 
tsMs of behavlQcal indicators seass to help vy stodents tetter 
Mter stand the r— i pt of fair verbal behavior. 

15. Umamdng the Intanctimi diagrau did not add anything to the unit 
far the stiadents In this clas. 

16. It «aa worthwhile for students to work in gravps and enant role- 
plaidAg dial n^Mi between s f ais nt s and teachera who are and 
aatfatlr . 

17. Tha suggestions for teaching procedures are claarlv desslbad in 
the pslde. 

18. I fait a need far more background to answer atoAent questions. 

19. This protocol wit needs to be better organized. 

20. The objectives cf the protocol unit were made clear to me by the 
lnaesactor*s gutea. 

21. I will use this protocol unit figain in my education courses. 

22. This protocol unit needs major revision before further use. 

23. In the space belaw dlscnss the weakest feature cf this pratacol 
umM. and iihat can be ilane to imprave it. 



24. Any oslier cotnmenta? 
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STUDEirr BACKGROUND IhTCKMATION ^nd Pre-^test (following) 
lutructlozis: The purpose of this questionnaire is to gather Infomation 
OD ttttdents who coaplete University of Colorado Protocol Materials 
unlta. Your responses will be kept confidential and be used otay to 
evaluate the materials. You will need an identification nuaber. We 
suggest you use your Social Security number or your student (Mtrlcu- 
latlon) number. Whatever nuizber ycu use, IT IS VERY LMPOP.TAN T THAT YOO 
USE THE SAME :?U?!BER ON EACH QUESTION-rrAIRE YOU CO:iPLETE FOR THIS PROJECT. 

1. Student Identification number 

2. Circle one: M F 

3* Grade Point Average (approxiaate) 

4. Age ^ 

5. What la your undergraduate major? 

6. Do you plan to become a teacher? 

Circle one: Yes No Undecided 

7. Have you had any previous teaching experience? Yes Na 
8« What aobject did you teach or plan to teach? 



9. What level did you teach or plan to teach? (Circle one letter) 
A. Preschool B, Elementary (K-6) C. Secondary (7-12) 
D« JIanlar college £• College 

10. Approxl&ate number of semester /quarter hours in college education courses: 

- Number of semester hours 

- Number of quarter hours 

11. What is your teaching status during this semester /quarter? (Circle one letter) 

A. I am teaching now • 

B. I am atudent teaching* 

C« I have completed my student teaching. 
D« I plan to student teach at a later date. 
E. Other 

12. Describe your present acadecilc status: 

A« Preshsan 

B. Sophomore 

C. Junior 

D. Senior 

E. Have cociplet&d bachelor's dcsvee. 
P. Have completed master's degree. 

13. Have you had experience with children? 

Caap counselor Sunday school teacher 

^. Playground supei-vlsor My ovm children 



ERIC School bus driver Other (Please describe) 



Protocol Materials Dcvelopmert Project 
Unlveralty of Colorado 

QUESTIONNAIK£ 
for 

Concepts About Teaching 

USE THE SAME IIUKBER YOU 
USED ON OTHER QUESTIONNAIRES 

IDENTIFICATION NUMBER 
PART I 

Instructions: Please Indicate your degree of agreement with each statement* 
Mark your responses In the space provided before each question, 

SA ■ Strongly agree A • Agree N • No opinion D » Disagree 
8D " Strongly disagree 

1, It Is all right to correct students when necessary, as long as 

they are praised vlien they deserve It, 

2. The factors Influencing an Instructional situation usually can 

be identified, 

3, If you criticise students severely for minor things, then you 
""""" probably won't have any major discipline problems, 

A, Teaching in ways to facilitate and improve mcaniug is a basic 

objective of instruction, 

5, One difference that can be established aiaong teachers is the types 

of questions they ask, 

6, Student involvement is higher in small group discussions than 

in discussions involving the whole class* 

7, Selecting instructional activities involves straightforward 
intellectual decisions based on the subject matter, 

8, The classes I teach should be more enjoyable than most classes, 

9* Informntlon about specific characteristics of students is impos- 
sible to find, 

10, Keeping order in the classroom takes priority over pursuit of a 

task, 

1 1, The foctors influencing classroom situatlont: can be classified into 
larger components or categories* 

1 2, Students are used to inccnslntent feedback and don't give it much 
thought when they arc praised or corrected for no apparent reason, 

^13. Field trips are all right for elementary school, but not for high 

school. 

l A, Teacher questions are basically alike, but some student answers 
are better than others* 

1 5, I don't' have time to worry about how my specific behaviors influ- 
ence student learning* 

^16, There aren't very many choicer available to a toachor when it coues 

to selecting an instructional progi ai.i for students, 

EKLC 



_17. It is more Important to teach facts than to spend tine on de- 
riving Interpretations and generalizations from facts. 

_18. Most scxidents accept school as a necessary and valuable exper-^ 
lence of their life. 

_19. A teacher should not allow students to get away with things 
that bother the teacher. 

_20. It Is useful to analyze quest lon-ansver-response sequences 
between teachers and students. 

_21. Soall group work In the classroon tends to be a waste of tine. 

^22. Students tend to respond In ways that reflect the teacher's 
expectations. 

_23* Responses or comments teachers make are as Inportant as the 
questions they ask. 

_24. A teacher should let students do things their own way. 

25. All students are basically alike. 

26. Praising students who don*t deserve It Is better than never 
praising anybody. 

27. It's much easier to be a good teacher than it Is to be a good 
doctor. 

28. Decisions about Instruction should take into consideration the 
needs of individual students. 

29. Students have to create their own knowledge; a teacher cannot 
give it to them. 

30. The only way to learn to be a good teacher is to teach for while. 

31. I have trouble visualizing how the things I read and hear about 
in education courses would actually influence a particular class- 
room. 

32. Visual aids are an unnecessary bother when your objective is 
teaching concepts. 

33. It is better to know nothing about a student's background and 
school records because such information is likely to prejudice 
your teaching. 

34. It is more important for a teacher to ask a lot of questions than 
to select certain kinds of questions to ask. 

35. Since the best teachers arc "born and not made'\ education courses 
are a waste of time. 

36. Students should be asked to apply concepts or principles to un- 
familiar situations. 

37. Irrelevant exaiiiples or non-examples should be avoided when teaching 
a concept. 

38. Students should be praised, even when they don't deserve itf for 
notivational purposes. 



^9. Analyzing classroom beliavlors, Including the teacher^s, Is 

an important part of teaching. 

^0. Field trips are valuable experiences for students. 

*1« Students learn in their own way in spite of what the teacher 

does. 

^2. Every classroom is so different, it's impossible to generalize 

about the instructional process. 

^*3. The sequence of mental activities performed by students cannot 

and should not be controlled by the teacher. 

What I have learned in my education courses won't matter when 

1 get out into an actual classroom. 

45* Most things that happen in the classroom influence the total 

situation in some way. 

^^6. Analyzing classroom situations is a necessary part of teaching. 

♦7. As long as everything seems to be all right in my classroom, it 

doesn't matter if 1 understand exactly what's happening. 

.^8. Viewing and analyzing films of classroom situations won't help 

me be a better teacher. 

PART II 

1. An English teacher decides that the students in her class should all 
read and analyze David Copperfiel d. What given conditions would have 
the most bearing on the success or failure of her decision? Rank order 
the following lists 

Reading level of the students. 

Student attitude toward school. 

Subject matter competence of the teacher. 

Socio-economJc background of the students. 

Atmosphere of the community. 

Students' previous grades in English. 

Scheduling of the class. 

Personality of the teacher. 

Sex of the students. 

Students' level of knowledge in subject matter. 



Piirt II 



2. 



}tmmm thrmm ccNicrptt ot imrmB each of irhUh %^ld lnclu«lv«ly 
d«Mrlb« all tha f#ctor0 on th^ lint b#lov on tho right. 



1. 



toactwir 

loach#r asking qu#ationa 
taachor praising sttidanta 
studtnts 

stiitfants roadlng books 

studants «#orking in sauill groupt 

•ovla projactor 

•ovaablr dask 

blackboard 



2. 



). ChooNO or ^ of the concapta or ioraa you hava liatod abova and dlvlda 
it Into conponant parts. 



concept 
teru 




4. Wrlia a aantrnco or tvo ihAt drscribt s the fuMtional ralatlonahip you 
hava diagroMicd. Co lurihor than aayina aoMthlng Is subMaod by lottc- 
thing alaa or thst soMthlng Is a part of aoMlhine alaa. Analyse tha 
ralatlonahipa you llluatrstcd and daacriba It, 
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Which unit of anAlysls would be b«8t for otstrvlog the follot^nR 
verbal bthjviors. Respond by placing the Utttr of the unit of 
analysis in tha spaca provldad. 

1. logical exposition in a 

sonologuu 

study drill 



2. 
3. 

4. 

5. 
6. 
7. 
S. 



typas of quaationa a tcachar 
asks 

appropriatanass of teacher 
feedback 

classroom cliaate 

kind o( student participation 

aoall group diacuaslona 

teacher influence on atudent 
level of thought 



A • quest ion-ansvor-response 

B. one-nlnute interval 

C. quo fft ion-answer 

D. specific cooncnts 



claesifiad by type 



Wliat dlffertiicae would you obaervc between an open teacher and a 
clo»^d teacher iu verbal interaction in the classrocu? Use your 
analyNlr of the following node] as a guide and syntheslxa or euiaiia- 
rise your thoughte into one or two sentiences. 



Teacher 
Percept iors 
and 

Paraonallty 



Inf luenc 



S5> 



Teacher 
Claaflrooa 
Orientation 



inf luenrci 



Kinds of 




Teacher 


Jnf Juencc , 


Questions 


or CoBrrcnts 





Learner 

Kf^havior 



Inf luencei 



Teacher 
Response 
to Learners 



influence 



Laarnlng 

OutCOBCS 



ERIC 
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7. You %ri8h to develop the concept ^'assets** for students In your book- 
keeping clasA* Number In order of occurrence the verbal statements 
you would make* (You nay assume Intervening student responses.) 
Mark a minus (-*) by those statements that are Irrelevant to teaching 
the concept* 

Jane, are Iho shoes you are wearing one of your assets? 

^ Why isn't this blackboard one of my assets? 

Is your father's car your own asset? 

Assets can bo defined as "anything that makes up the entire 

properly of a person or a business/* 

Free enterprise is one of the basic principles of capitalist* 

Cnn you think of a liability you have? 

8. A teacher in a middle class suburban community has 35 seventh graders 
for 50 mJnutc periods each day. She has never taught before and has 
a B.A. In Hicitory. She wants to Improve her students' attitudes 
toward school » but she has to teach State History* Rate the following 
Instructional alternatives as good, average, bad. 

students give oral reports 

students construct a time line mural 

^ students do research and work in snail groups 

teacher leads discussion 

each student rends n book on the history of the state 

students watch a 60-inlnute movie of excellent calibre 

studcutr. write pnpors 

students make notebooks 

adapt net Ivltlon from national currlculun projects 

teacher lectures 

_______ teacher has radical speakers on mistakes the state has made 

students make scale uodels of how their state grew 

9. Kame five ways to find out information about the characteristics of 
learners. 

U 

2. 

y. 

A. 

5. 
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10. Check the verbal behaviors which are appropriate and pertinent In 
teaching for concept attainment. 

The teacher should Indicate the accuracy or Inaccuracy of the 

student response. 

The teacher should Indicate a contrast betveen examples and 

non-examples of the category. 

The teacher should help the students formulate a definition 

of the category. 

The teacher should avoid asking leading or probing questions. 

The teacher should avoid giving examples that do not meet 

criteria for the concept. 

The teacher should ask the students to list and group things » 

events, or ideas that characterize the concept. 

11. Two teachers, X and Y» were observed in their Interactions v;lth 
seventh graders. X was observed Instructing tracked classes A and C 
and Y was observed instructing classes In tracks A and B. Track A is 
the highest achievement group and Track C, the lowest. Observers con- 
cluded that these teachers did not differ their classroom behavior in 
working with students of different achievement levels. YOU ARE TEACHER 
••Z'* AND TEACH ONE TRACK "A" CLASS AND ONE TRACK "C" CLASS IN YOUR 
SUBJECT MATTER AREA. 

A) What do you think the conclusions derived from observations of 
your class would be? 



B) If your behavior differed between the two classes* describe 
specifically how it differed. 



ERIC 
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Underline any teacher statement in the follovring transcript that you 
think might indicate that the teacher has a strategy for organising 
facts to teach the concept, writ of mandamus > 

Transcript 

Teacher: One of the first and most Important court decisions was 

Mgrbury versus Madison , Can anyone tell us details of the 
case? 

Student: Marbury received an appointment from Jefferson, but when 

Madison became president, he refused to sign it and Marbury 
sued him and the court said OK. 

Teacher: Very good! What was that court order called? 

Student : A writ of mandamus . 

Teacher: Fine! What does that mean? 

Student: You got to produce the body in court. 

Teacher: No, That's a writ of habeas corpus . Now what is the 
difference? 

Student: Well, the court can order Presidents or anybody to do any-- 
thing to enforce lav. 

Teacher: All right. What principle of American government does that 
illustrate? 

Student: I don't know. 

Teacher: The separation of powers between the courts and the Presl-- 
dent. 

Student: 0^ the federal-state relationship? 

Teacher* The case of Marbury versus MAdison Is one of the most famous 
in American history becauce the writ of mandamus order showed 
tliat thero is a system of checks and balances that works. 



13. Analyse the diagr«m. Describe the type of feedback thlt teacher 
la giving In the space below. (150 words) 



Sue 



Sally 
/> A . A A 



V ^ V 



Teacher 



^ 

^ 

< 

-> David 

-> 

< 

-> 



/X ^ 



Harry 



positive feedback 
corrective feedback 



correct response or behavior 
Incorrect respooee or behavior 



ERIC 
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14. G€Qtrallse the difference between the verbal behaviors of Teachers 
A and B In relation to the coemionts or actlvitea of the three 
students. Sherry » George » and Dick. 



George : 

Teacher A: 
Teacher B: 

Dick: 

Teacher A: 

Teacher B: 

Sherry: 
Teacher A; 

'ioacher B: 
Sherry: 
Teacher A: 
Teacher B: 

Dick: 

Teacher A: 
Teacher B: 



When a person feels threatened » he is likely to oaka 
a strong or violent response. 

That's a massive generalisation on your part. 

Can you give some oxnmples that led you to that con- 
clusion? 

(Reaching in to the aquarium to touch a fish) Gee, 
what kind is this one? 

Get your hands out of there I Vye told you all not to 
touch that aquarium. 

That's an Dnperor fich, but it will hurt him to touch 
hla. 

I think that vapor condenses when it hits a warm surface 

That's a good gueos. Isn't anyone else willing to try 
like Sherry? 

Tell me vhcrc you have seen that happen? 

(Feeding fish without asking.) 

That's very thoughtful of you. Sherry. 

Wc have a schedule for feeding the fish. Check with 
me before you feed them. 

One reason not many explorers caac before Columbus did 
was because their ships weren't that good then. 

Where were you when we talked about the Vikings? 

Pursue that idea; It's a good one. 
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Describe In three different ways a teacher who uses fair and unfair 
verbal behavior. Choose one from Column A and one from Column B to 
construct each > r responses. 



Fair verba: • ./ lor 



1. 
2. 
3. 



Column A 

1. Praises 

2. Gives In to 

3. Corrects 

4. Does not correct 

5. Does not praise 

6. Does not give in to 



Unfair verbal behavior 

1. 

2. 

3. 



Column B 

A. student behaviors that don*t deserve it. 

B. some student behaviors when they deserve 
It. 

C. Ideas or actions not students personally. 

D. student behaviors when it Is not clear 
whether they deserve It or not. 

E. student behaviors when they deserve itt 

F. some student behaviors when they don't 
deserve it. 
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Read the transcript on the last tvo pages (pp. 14-15 ) and use it 
for your response to the following questions: 

A. Which teacher question, regardless of the actual student response, 
would be likely to elicit the highest level of thought In the stu- 
dent response? 

a. 1 

b. 3 

c. 5 

d. 9 

e. None of the above 

B. Which student response is at a higher level of thinking than the 
others? 

a. 2 

b. 4 

c. 6 

d. 10 

e. None of the above 

C. Which teacher question, regardless of the actual student response, 
would be likely to elicit the highest level of thought in the stu- 
dent response? 

a. 7 

b. 11 

c. 13 

d. 14 

e. 16 

D. Which student responses can be considered to be above memory level? 

a. 10 

b. 12 

c. 15 

d. 17 

e. None of the above 

E. Which teacher question did elicit the highest level of thinking in 
student response? 

a. 18 

b. 20 

c. 22 

d. 24 

e. None of the above 

F. Which teacher question is least likely to elicit a high level of 
thought in the student response? 

a. 18 

b. 22 

c. 24 

d. 25 

e. 27 
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Which student response is most clearly representative of a 
setBory level response? 

a. 23 

b. 26 

c. 28 

d. 30 

e. None of the above 

Approximately what number of the teacher cpiestlons generally 
encourage (enable, support) students? 



Given your answer in 'H", would you have hypothesized that the 
number of student responses that can be considered above memory 
level would be; 

a. Greater than it is 

b. About the same as it is 

c. Fewer than it is 
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TRANSCRIPT for QUESTION 



Teacher : 



Steven : 
Teacher : 
Steven: 
Teacher : 
Steven: 
Teacher : 
Steven: 
Teacher : 

Steven: 
Teacher: 
Steven: 
Teacher : 



Paul: 



Teacher : 



Brent : 



Okay, let's talk about the story of the auccess of Little 
Black Nose. Who can tell us who Little Black Nose vas ?^ 
Steven? 

2 

A steam engine. 

3 

Can you tell me a little bit more about him ? 
No.* 

Where did he come from ?^ 
6 

America. 

7 

America . Can you tcIJ roe about hiu naioe ^' 
No.® 

We have two names, the title of the story, "Little Black 

9 

Nose", and In the story Lhere wa s f. y;thev nr^me . Steven? 
The DeWltt Clinton. 

Do you remember vho DeVJltt Clinton vas ?^^ 

12 

Wasn't he a famous American? 

Yes, he was a famous statesman, wasn't he. How many of you 

can think of seme things that have been named for famou s 

people ? Think of soraethlng that has somebody else's name 

14 

on It, that has been named after somebody . Paul? 

Well, there's a ship that's called The Lincoln, named after 



Abraham Lincoln. 
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The Lincoln, Okay. Can you think of something else named 
after soinc-body ?^^ Brent ? 

Well, there's the Bell Telephone Company named after Alexander 



Graham Bell. 
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TMCher : 

Dvlght : 
Teacher : 
Dfflght: 
Teacher : 
Ihrlght: 
Teacher : 



Nlel: 
Teacher : 
Nlel: 
Teacher : 
Nlel: 



Very good. Another thing thct'a been named after soaebody , 
DvlRht ?^^ 

Our school ±B named after <^om«tie. 

20 



19 



22 



Alright y It was named after > • . 
John Addxae. 

Can you tell me sotaething about John Adeims? 

23 

He vae the second prea'cien^-^"^ 

OA 

That'e right 1 vrny ,i<^ vf . tl^rngs after people? Why do 

e n to ship^ and schools and 

Inventions? 



yoi: suppose vtf L'A^^_ 
.25 



Welly V73 wouldn't pick jlucf; anybody's name for these things. 
Why not ?^^ 

28 

Well, we inlj^ht want to repii\'>cint something. 
Can you odd on that» Mc lV 

Well, maybe ths guy they natac souiethlng after is soijebody we 
should remeaiber, or \aaybe he \:as one of the guys who helped 
invent it*"^^ 



26 
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Protocol MatcTinls Development Project 
University of Colorado 



STUDKNT EVALUATION QUBSTIOVNAIRE 
for 

Conceptualizing the Process of Instruction 

USE THE SAME NUMBER YOU 

— ^ USED ON OTHER QUESTIONNAIRES 

IDENTIFICATIOJ; NUMBER 

Instructions: Indicate your degree of agreement with each statement. Mark 
your responses in the space provided before each question. 

SA - Strongly agree A = Agree N • No opinion D « Disagree 
SD « Strongly disagree 

1- The concepts dealt with in this protocol are worthwhile for 

teacher preparation. 

2. Compared to what normally takes place in my education courses, 

this protocol unit was more interesting. 

3. I have loarned less from this protocol than from other activities 

in education courses. 

I fovnd the film to be pretty boring- 

5. The picture quality of the film needs improvement. 

6. The audio quality of the film needs improvement. 

7- Listing observations from the first two episodes of the film 

was v;orthwhilc. 

^* Working in small groups to classify our observations from the 

film was worthwhile. 

9- Having; transcripts of the classroom episodes in the film for 

reference was very helpful. 

^0- Drawing a model of the process of instruction wasn't a very 

significant activity. 

11. Writing, a descriptive analysis of the last episode in the film 

was a worthwhile activity. 

12. The content of the three classroom episodes in the film needs to 

be more varied. 

13. It botliered me that all three eplaoJes in the film depicted social 

studies classrooms . 

^14. This protocol unit needs laajor revisions before further use. 

15. In the space below discuss the weakest feature of this protocol 
unit and what can he done to improve it. Use the hack of the 
page if you need more space. 
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Protocol Materials Development Project 
University of Colorado 

STUDENT EVALUATION QUESTIONNAIRE 
for 

Learners and Their Characteristics: 
Implications for Instructional Decision-Making 

USE THE SAME NUMBER YOU 

USED ON OTHER QUESTIONNAIRES 

IDENTIFICATION laJMBER 

Instructions: Indicate your degree of agreement with each statement, Mark 
your responses in the space provided before each question. 

SA « Strongly agree A - Agree N « :;o opinion D « Disagree 
SD « Strongly disagree 

!• The concepts dealt with in this protocol unit are worthwhile for 

teacher preparation. 

2. Compared to what normally takns place in my education courses^ this 

protocol unit was more Interesting to me. 

3. I have learned less from this protocol unit than fror* other acti- 
vities in my education courses. 

The slides on "Student Attitudes Toward School" were interesting. 

5. The quality of the slides on "Student Attitudes Toward School" needs 

linprovciiiant . 

^» The slides on "Student Attitudes Toward School" stimulated a signi- 
ficant discussion. 

7. Working with the student profiles was worthwhile. 

8« Working with the student profiles was boring. 

9. The quality of the slides on "Instructional Alternatives" needs 

Improvt^ment . 

10. The audio quality of the cassette tape accompanying the slides on 

"instructional Alternatives" needs improvement. 

^11 • The slide tape on "instructional Alternatives" wasn't very informative. 

^12. The slide tape on "Instructional Alternatives" was interesting. 

13 • Working with the student profiles following the slide tape on 

"instructional Alternatives" was worthwhile. 

___14. This protocol unit needs major revisions before further use. 

15. In the space below discuss the weakest feature of this protocol unit 
and what can be done to improve it. Use the back of the page if 
you need more space. 
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tr«tocol Materials Pcvlop— m ^reject 
lhilv«rsll]r of Colorado 

tlVPDIT CVALUATIOSI QUESTIomuiU 
tot 

Vortei InC«racCloii in ch« CotnlClv* DlMtioleot 
TiM loUtlooohir koCMTOO Toochor Vorb«l lohavlor ttwdooC Ro*^«o 

on m lAME Himici tou 
otip oil onwt qutrrioRiuius 

IDBfririCATlON KUMin 

iMCmcClooa: Xndlcot* your 4«cf»< of ^vmtkt vicli oocli oUtflMnt. ftork 
yoor roapoMMO In the opocc* provldod ^fero oocli ^ocloo. 

tA • tirontijr Affroo A - Agroo K - Ko of iaioa » - DlMfroo 
fli • Stroatiy 4itktt— 

i« Tko coneo^to doolt with in chlo protocol oro wortlMillo for 

toockor ftfrntrntiom* 

2. Coaparod to tdoit nortially Cokoo pJocr 1« ay oducocloo courooo. 
iklo protocol ttnlc wen ror« Inltroot lug. 

). I hay* Uarnad loa* C^ra thl* protocol onlt than froa oth«r acti- 
vlcloa In oducatloo coura**. 

4. I foMid tha rila CO bo pretty Wring. 

S« Tlio plctora ^lity of tho flla oooda laprovoMoMt. 

4. Tho Mdlo ^licy of tha flla oooda liaprovaMot. 

7. TW diodtaelon of tho aiulytleal franowoffk dovolopod by nacdoaald 

•■d torot Mooii't vary ■Igniflcoat. 

i« Motclilat tko firac two apiaodoa of tiM f 11« and Mrkiog tlio 

Mtrlcoo MO MorclMhlU. 

aworprlot cImc dra.*rlbad th» iMotactioo U tho cUaarooa «m 

COttftMUg. 

^ >»rpaaa of prod let log clw rotpooooo im oplaodo ckrao waa oavar 
vory <loar to m. 

11* iovlog tranacrfpta of ttia rlaaarooa oplaodaa In tlio f 11a for 

rofor«^ieo oaa vary halrful. 

il» Tkla aalt ««•« taachlng X\w Haedoaald-Iorot fraaowark. rathar tkoa 
focaotMt oo oao wy to look at varUl iMoractUa. 

_19. TIm ikroo opUodoa of tha flla lllootratod tha cognltlva dtooMloa 
of varkol latoractioo. 

_1A. Tkta protocol vale n««Ja u«Jor rovlaloaa kofora Ita aoo. 

II. U Kk* •»>«4;« kolow dlaCMao tU Ifoalrat faatoia of ftkJt protocol 
«mU aod idiat ran ko dona to Inprovo It. tfao tho ka«k «»r tko 
pofo If ff^ Mod —4 9 vporo. 
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Frotocol ltat*rUla D«v«lopMnt freject 
VUwrtUy of Col«r«4« 



nwhvr ivALUATicxi quESTtonuiu 

for 

OrgMlgUc Pacta 
T« Taack Namlafful klatlmahlpa 



Pt l Vm U>il WW TOO USD 

ov omt nuiiTiiwmmi 

lastffMtiMiat Mleata jrwir tfatraa of agroMMK with oack aC«K«M»t. Iterk 
yMT raapa— »■ im tha mfmcm rrovl4o4 Wforo OMk ^atioa. 



* ttrMfljr atTM A • Acrao R • Mo opUlo* 9 • Biaaaraa 

- 4U 



1. tlw cMM«rt« 4oalK irlKk la kkia protocol ara wartkiAlla U 
ek»r propatar loo. 



2. Caspar*': to n or<Mlljr caV»« pl«.» la ay o>locoKloo coMraoa, 
Kkta protosti . .r ^a aora latoraaKlnB* 

I iMTa loaraa^ laaa fra« KkIa protocol aolt tkaa fraa atkar actl- 
vltlaa U aia e atloo couraaa. 



.4. I foMi tha flbi to ka prattjr korli«, 

.3. Tka plctora foallty of tka flla 

. TW aotflo ^Utjr of tka flla oaotfa laiprovaaaat . 

_ 7. 1W aokjoct of tka looaoo, coocapt taaaklaf, waa too caaplicatri. 

. t. Hm Mark m 4«f laltlooa aacaaaary. 

. t. Tkara mm tao ooak kolp givoo oo coMtmctlag tko vorkal iailcatora 
of c at ap t taackiiif . 

JO. Hm fila akauli kava koao akow oarllor la tka aait. 

.11. Aaalpalat tka traaoeripta of tka tao aplaoCoa aaa vartkMktU. 



11. Maklat • liM of aoa-asaaploa of vorkal latfiaotora fraa CU 
#f aaa aartkaklla. 



.11. Tka tflfforaaca* la tooaklag arratofiaa of tka too taackara la tka 
flla wro tflfflcMlt to aao. 



Ik. IhU protocol uait aoo^a aajor rovlataaa kofora itm aao. 
IS. la tko apaca krla»r 4lac«aa tka %*aa k aat faatora of ckfa pracacol oaft 
aaotf^aar *^ ^ ^***' ^'^^ *** H«* if 
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rr«Cocol NttcrlAl* DcvelopKent Froject 
ltalv«r«lty of Colorado 

SIVDEIIT EVALUATIOM qUESTIONNAIU 
for 

Pair Vorbol iohavior 

USE THE SAME NUMBEX TOU USED 

iDOTif icATiow mmm ^ <WTioinuiitEs 

iMCHMtlovot ladlcato your 4«troo of agrooMiC with —ch •ucwoqc. Hark 
ymr rooponooo i» cho oimco provldod boforo oach ^weodon. 

1. l%m CMcoyco «oalc irlcli In ckla rrotoeol aro MDrclnrfillo for 

toaakor prafaratloa. 

2. 



CMparad to what aorMlly take. pUca la my oducaCloo couroaa. 
Ckla rrotoeol unlr waa 9or* lataraatlnc. 



J. I kava loaraad laaa fro. this protocol ckaa fro* oCkar actlvleiaa 
la adueatloa courara. 

4. I faiMj ckt flla to ba pratty U»rii«. 

5. Tka plctura ^llcy of cba fll« naoda laprovaMt. 

Tka aiidto qaellty of cha flU naoda laprovMaat. 

I. Tkla color 111. waa kaCCrr aad morm intaraaClag than cha aana fila 
MMiJd kava kaan la black aad wkica. 

IW ^atlaM aafcad darlag tka fll. dUa'c aarva any aaafal purpoaa. 
' bTb^ikla!' ^'^ -^O"* to 

.10. Tka dlacMalaa follovlag tkf> fil« ««a»'c vary alsaificaac. 

trlSiS t***" cUaataa. eplaodaa U tka fil. in tar«a 

I'.^fTT'L"'*^'"**" ^•r^t^in^ af tka cai«apt - 

fair Varkal laliavlor. 

Sis?!*?! Jr!*'*!?**" <l»r.ra«a of tka tvo cU.area..a did not add 



ISiS.^ir;," JT'U"^* ^^T rflaloguaa bat,*.ca acudanta and 
taaakara wko at* fair and art faic «aa vortkakila. 



.U. Tkia protarol anit aaada aajor raviaioa bafara fartkor aaa. 

15. U tka w^m balm, diacaaa rka aaafcoat faatara of tkia protocol anit 
and ««Uat can k» dona to toprova It. Uaa tka back of cl.* paaa if yea 
naad uora aaac#. '^•^ 
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Crlterlon Test - Conceptualizing the Process of Instruction 

nsnncTioNs to oiSEtm 

Tott alrMdy hmw had aperlencc m» gcudeiits In classrooos. and 

IT-ir* TT "f*"/*?*^ goaa on in an Inatructlonal situation. 

AM ytm mteh a^h of cha ahort taachlng apiaodaa in tha flla, wrlta 
law your apacifle ebaanratlona about uhat la happaning In that claaa- 

EPIMOI #1: 



■PIMM tit 



1 
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Criterion Test — Verbal InLeraction In the Co£ntttve Dimension 

(teat is epaiode 3 — next page) 
Pleaae use the Mt rices provided below to claaaify the verbal behavlora 
in the three daaarooai epiaodea preaented in the fila. Tally each atate* 
Mnt either aa a productive or reproductive reaponae on the part of a 
atudent; or aa a tranaact ion-oriented or a role-*expectancy oriented rea-* 
ponae on the part of the teacher. 



EPISODE ONE: 

Teacher 

Tr anaaet ion*-Or ien ted 
Deciaiona 

SegiMnt 1 

SegiMnt 2 

SefAent 3 

SegMnt 4 



Opening 

A 



Student 
Productive 
Behavior 



Segnent 1 

Segaent 2 

Seginent 3 

SegMnt 4 

Role-Eicpectancy 
Oriented Deciaiona 



Cloaing 



Reproductive 
Behavior 



EPISODE TW): 
Teacher 



Tranaaction«>Oriented 
Deciaiona 

SegMnt 1 

SegMnt 2 

Segiient 3 



Opening 



Student 
Productive 
Behavior 



Segiienr 1 

Segwnt 2 

Scgpent 3 

Role*Expectancy 
Oriented Deciaiona 



V 

Cloaing 



Reproductive 
Behavior 



EPa30DE THREE: 

TranMctlon*Orl€nt€d 
tecl«lon« 

SttgMnt 1 

S^gMnt 2 

S«tMOt 3 



Opening 

A 



Student 
Productive 
Behavior 



Segnent 1 
tegpent 2 

Segment 3 

Role- Expec tency 
Oriented Decisions 



Cloelng 



Reproductive 
Behavior 



1. Predicted reeponee to **Can he do that?" 



2. Predicted response to **Hov do ve get to this?** 



3* CoMente on verbal Intoroctlon rclationNhlps after viewing all throe 
segMnte of Episode 13. 



I 
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Unlvcrf.ity of Colou io 

"^2- Post-leal 

Ql'KSTro::;:AiK!: 
for 

Concepts Ahoul Tonchinf, 

USE Tl!i: S.V'.y. MT.'.IR VOLT 

'^uiiiiTii^iaaToin—iH-- ^^^^ ••■ aikks 

Inutrucllonr.: J'U. vr Indlc.it.- your dcrroo of .irr.c • ...»» . , 

1- It i« nlJ rifiht to corrc^ct sluJ.-wis v;ho„ nccc5f::.ry, a. ]our as 

they .ire iUfliscH vhou t lu-y Jcsivvi- it. ^' 

^' "'^'^^"^'^ srvorcly for miner ihlr.r.s, tlun you 

6. StuJr,U 5..;olv..c.t is hi,'..- m r-nU ,roM, c! 1 ccu.r.ions th->,- 

an oist-iii .-.0.1...... lavolvin- the yhc^"* c cli.r.a. 

7. Stjcctii.r lr.':lrrc;Jo:,n] nctivili^ ., involve-, > . i^ht^or- r^' 

Intcllc.clu.l c-c-ciclon = , l.sc.i on l!.. .uLj,cl r.ltcr. 

8. The cL-.r..c.s I rc.ch .held U- r.n o cnjoyabU- fr u.ucrt c],-...c... 

^^mnrfi.d:''" of .tuc:.u. I. 

lac'!"" ^" Priority over i^ur^.ti of a 

11. Tho factors -,riuc.ncinr. c] a:;:;rcK..:. t.'tu.ntir.ns c.n be c].i:.ifjf.j ivvo 

laificr cor;: . , ntr cr catcjori. ti. fi-^iJJf-u j,uo 

12. Sti.d..n . nre urcd m li.corr. i.^tc.L f c dbnr.!: .v.u' don't i;Jv: it ^..ch 

thou£lu vhcn they arc Kr.3r..d or corrected f c . no ..pp'rc-ut roa'ca. 

rchiol!"'"' "'^ ''''''' ^•^^■^^"^'"^y but not for hlM. 
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_17* It Is more i-portant to Lcncli farts th.in to cpcml t i;no on dc- 
rlvinc inti-riTiaal Ions ami [;( tki al i /.alions fvm facts. 

^18. Most ctudcnts accept school as a nccoKsary nnd valuciblc c:;pcr- 
lencc of tliclr life. 

_19. A teacher sliould not allow students to get away with things 
that bother tho Icaclier. 

^20. It is ur.cful to ana1y::e quest ion-annwor-rcr»pon5*o sequencer 
between te.Ti hcrr. and ctudrnts. 

_21. Sn:all croup worl; In ihe classrociii tendr to be a v.Msto of tine. 

_22. Students tend to respond In wayn thil reflect the teacher's 
cxpcctatlono. 

_23. Rer.pont;t?s or co-^'ionts teachers nake are n« iinportant as the 
quest ionr. they ask. 

_2A. A teacher should let etude nt;j do ihin^ri their ovni way. 

_25. All studentc are basically a] Ike. 

^26. Praicinc rtudiiilf: who doii'l dopervf it Is better than never 
pvalsinf*, anybody. 

_27. It's ruch easier to be a (;ood teacher than Jt ir. to be a fooJ 

doctor. 
r 

_28. Decir.ions about irrtrurtic^n should lake Into considc^rnt Jon i ho 
needs of lndJvidu:il t:ti:di .-.ts. 

_29. Students havt- to create their ovrn knowledge; a t<.aclier cnnnol 
pive it to then. 

_30. The only way to Ic.-irn to be a good ttachor is to teach for v'niJe. 

_31. 1 have trouble vJ ruuH ; l.ow tho thi;ir,r* I rearl and he-ir r! j it 

in education coui>;er, v.ovilcl acrurilly influence a ir.rticulcu dr.:.;- 
rooa. 

32, VJbual aids are an unneco'^fary bother when your objective Is 
teach Jnj; concepts. 

_33. Jt lo better to knov; nothing*, about a student's background and 
school records locause cuch Information is likely to prejudice 
yoiir tcachlnc. 

Zk. It is more important for a teacher to ask a lot of questions than 
to eelcct certain kinds of quest lon.s to ask. 

^35, Since tho best tcachors arc "born and not made**, education courr.cs 
are a waste of tiuie, 

^36, Students should be asked to apply concepts or principles to un- 
familiar situations. 

yi . Irrelevant cKanplcs or non-examples should be avoided v;hen teacliiR^; 
a concept. 

_38. Students should be prai<K*d, even when they don't deserve it, for 
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_39. Analy^cJng cl/j?5f;roc:n behaviors. Including the tcachcr'e, Is 
an Inportant part of teaching. 

__A0. Field trips arc valuable experiences for students. 

Students learn in tljcir own way in spile of what the teacher 
docs. 

_A2. ^^fery c]a5;5;roo:n is so different, it^s impossible to generalize 
about the in5?tructionnl process. 

_A3. Tlic sequence of ncPiOl aclivitlc;; pcrforned by students cannot 
and should not be controlled by the teacher. 

^44. Wliat T have learned in r.y education courses won't matter when 
I get out into an actual clascroOiH, 

_A5. Most thinr.s that happen in the classroom influence the total 
situation In some vjay. 

_46. Analyzing clnssrooM situaMors is a necessary part of teaching. 

_A7. As long as everything seems to be all right In ny classrooLa^ it 
doecn^t natter if 1 understand exactly v/hat's happening. 

_48. Viewing and analyzing riltns of classroom situations won't help 
DO be a better teacher. 
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APPENDIX D 
Statistical Tabl 
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Conceptua lizing the Process nf T^.^rnrMnn 
Variable 1: identify, classify, and analyze variables in instruction 
Source MS df p 

1. Groups 1.83 2 ,33 

2. Covarlate 96.11 1 17.35 ^ ^ 

3. Error 5.54 

Adjusted Mean, Social Science Methods - 10.32 
Adjusted Mean, General Methods - 9.92 

Adjusted Mean. Comparison Group - 10.14 



Variable 2: Recognizing the importance of conceptualizing and 





analyzing variables 


in Instruc 


■;ion 




Source 


MS 


df 


F 




Groups 


13.66 


2 


3.30 


p< .05 


Covarlate 


181.57 


1 


43.78 


P <.oi 


Error 


4.15 


156 






Ad.lusted Mean, Social Science 


M<2thods K 


11.95 





Adjusted Mean, General Methods . 11.26 

Adjusted Mean, Comparison Group . 12.23 



ERIC 
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Learne ^aracteristics : Implications for Instructional n,...,- w,^ 

Variable 3: Recognizing and identifying individual differences 
in attitudes toward school 

Source MS p 

1. Groups .19 2 

2. Covariatc 167.03 i 40. A2 p^.Ql 

3. Error 4.13 155 
Adjusted Mean, Social Science Methods - 10.32 
Adjusted Mean, General Methods - 10.24 
Adjusted Mean, Comparison Group - 10.35 

Variable 4: Recognizing implications for instructional alternatives 
Source MS df F 

1. Groups 12.37 2 3.79 p<.05 

2. Covariate 241.89 1 74.03 p^.Ol 

3. Error 3.27 155 
Adjusted Mean, Social Science Methods - 9.67 
Adjusted Mean, General Methods - 10,54 
Adjusted Mean, Comparison Group > 10.89 
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Verbal Interaction In the Cognitive Dimension: 
ITie Relationship B etween Teacher Verbal Behavior and Student Response 

Variable 5: Recognizing differences In teacher verbal behavior 
Source MS df F 

1. Groups 29.64 2 8,A4 p < ,01 

2. Covarlate 645.82 ^ 183.86 p<.01 

3. Error 3*51 156 

Adjusted Ilean, Social Science Methods « 12.93 
Adjusted Mean, Gf^neral Methods » 11.38 

Adjusted Mean, Comparison Group » 11.41 



Variable 6: Identifying and analyzing student responses in 
relation to teacher expectations 
Source MS df F 

1. Groups 27.81 2 7.36 p<.01 

2. Covarlate 103.21 1 27.31 P<.01 

3. Error 3.78 156 

Adjusted Mean» Social Science Methods « 12.62 
Adjusted Mean» General Methods • 11.32 

Adjusted Mean, Comparison Group m 12.59 



EKLC 



-59- 



Organizing Facts to Teach Meanlnrful Relationships 

Variable 7: Identifying teacher behaviors that indicate teaching 
for concept attainment 
Source MS df F 
1- Groups 39.06 2 11.53 p<.01 

• Covarlate 321,87 1 95.00 P</oi 

3. Error 3.89 156 

Adjusted Mean, Social Science Methods -= 11,57 

Adjusted Mean» General Methods » 9,56 

Adjusted Mean, Comparison Group « 9.59 



Variable 8: Recognizing student behavior that Indicate concept 
learning 

Source MS df F 

1. Groups 8.39 2 2.67 

2. Covarlate 280.80 1 89.33 P< .01 

3. Error 3.14 156 

Adjusted Mean, Social Science Methods - 11.22 

Adjusted Mean, General Methods « 10.35 

Adjusted Mean, Comparison Group « 10. 4A 
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Fair Verbal t e havlor 

Variable 9: Differentiating between teacher verbal behaviors 
that are fair from those that are not fair 
Source MS df F 

1* Groups 36,01 2 8,07 p<,01 

2. Covariate 316.63 1 70,98 p < .01 

3. Error 4.46 156 

Adjusted Mcin, Social Science Methods « 11.32 
Adjusted Mean» General Methods « 11,32 

Adjusted Mean, Comparison Group « 12.77 



EKLC 



Variable 10: Understanding that teacher verbal behavior is 

related to cognitive, affective and social meanings 
communicated in the classroom 
Source MS df F 

1. Group 6.91 2 2.21 

2. Covariate 49.57 1 15.82 p < .01 

3. Error 3.13 156 

Adjusted Mean, Social Science Methr 7.08^ 
Adjusted Mean, General Methods « 6.87 

Adjusted Mean, Comparison Group " 7.55 




